Image enhancement using modulation strength map and modulation kernel

ABSTRACT

An image enhancement system may generate a net signed fractional modulation value at each location in an image using a modulation strength map (MSM) in combination with a modulation kernel F(θ). The system may scale an image value at each location using the net signed fractional modulation value at each location. Application of the algorithm may not result in systematic clipping of peri-edge locations to maximum or minimum brightness values, or any other form of systematic homogenization of values at peri-edge locations.

CROSS-REFERENCE TO RELATED APPLICATION

This application is based upon and claims priority to U.S. provisionalpatent application 61/644,935, entitled “Method for Increasing Salienceof Images/Videos to Improve Visual Function and Aesthetics,” filed May9, 2012. The entire content of this application is incorporated hereinby reference.

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH

This invention was made with government support under Grant No. EY016093awarded by the National Institutes for Health/National Eye Institute(NIH/NEI). The government has certain rights in the invention.

BACKGROUND

1. Technical Field

This disclosure relates to image enhancement technology.

2. Description of Related Art

Billions of digital images are taken every day. These are taken usingdevices such as handheld cameras and phones and video cameras runcontinuously in cities, buildings, factories, and cars. Trillions ofimages are already posted on the web for myriad business and personaluses. Given this pervasive and growing role of digital still and videoimages, technologies that can automatically boost the clarity and visualimpact of main objects in these images, both on screen and on paper, canbe of great economic significance. However, tools that automaticallyenhance clarity or salience of images may not target main objects, andmay not take best advantage of the cues that are known from perceptualstudies to increase apparent contrast between objects and theirsurrounds.

Human perception of natural objects is primarily based on contours,including occluding boundaries (where an object ends and a backgroundbegins), abrupt changes in surface orientation (such as where two facesof a cube meet), and major surface markings (such as the stripes on azebra). Artists have long known of the importance of contours for visualperception, and therefore use contours to represent objects and scenesin pen-and-ink line drawings. Line drawings involve only a tiny fractionof the original image “pixels,” but often contain all of the informationneeded to rapidly and reliably recognize objects and scenes. However,efforts to enhance images in a way that selectively and flexibly boostmain object contours can provide less than optimal results.

One approach to boosting images salience is to increase overallcontrast. However, every pixel in the image may be affected. This maylead to marked changes in image appearance. Shadows may be deepened,bright areas may be washed out, and colors may become unnaturallysaturated. At the same time, objects may not be emphasized.

Another approach is to selectively boost mid or high spatial frequencybands, as in “unsharp masking” and other sharpening approaches. SeeLeat, S. J., Omoruyi, G., Kennedy, A., and Jernigan, E. (1005), Genericand customized digital image enhancement filters for the visuallyimpaired, Vision Research, Vol. 45, No. 15, pages 1991-2007.doi:10.1016/j. visres. 2005.01.028. This approach may be used to boostdetails, rather than main objects, to counteract blur caused by lowquality lenses or loss of resolution during printing. Concerns withsharpening methods based on spatial frequency filtering may include thefixed relationship between the spatial scale and form of the local imagestructures that have been targeted by the filtering stage, and thespatial scale and form of the image enhancement itself. For example,when sharp localized structures are targeted in the filtering stage,image enhancement may also be sharp and localized. This may not lead toa desired perceptual effect. Similarly, there may be a simple, monotonicrelationship in sharpening between the strength of the image feature andthe strength of the image enhancement: stronger image features receivegreater enhancement. This may be the opposite of what is wanted: strongedges may not need enhancement but receive it anyway, and weak edges—forexample, places along an object boundary where the object blends intothe background and local contrast is lost—may need the most enhancementbut receive none.

Another approach to enhancing “objects” is the use of an edge detectionalgorithm to select image locations to boost. Edges may be the elementsof object contours. This technique has been used in “smart sharpening”algorithms. But once edges have been identified, the problem may remainthat the spatial scale and magnitude of the enhancement is tightlycoupled to the spatial scale and magnitude of the underlying imagestructures.

Once edges have been identified, an alternative use of edges is tosuperimpose lines on the original image where edges have been located.These lines can be either black, as in a traditional line drawing, or“bipolar” in which both black and white lines are superimposed in pairson the light and dark sides of the edge. See Peli, E. S. & Peli, T(1984), Image enhancement for the visually impaired. Opticalengineering, Vol. 23, No. 1, pages 047-051. While the width of thesuperimposed lines can be varied in such an algorithm according to thechoice of the user, the original image pixels are replaced by theenhancement. This may lead to loss of information and can produce anunnatural, cartoon-like appearance.

Another concern that can occur in both sharpening and superimposedline-based approaches is that, as the enhancement level is turned up andbegins to measurably increase the salience of the targeted structures,the processed images may begin to take on an undesirable appearance thatsome viewers may find cluttered or harsh.

The Razor Vision™ Video enhancement cable by Belkin (˜$200), follows theabove-mentioned filtering-based approach. However, it indiscriminatelyboosts contrast within a specific spatial frequency range. This mayhighlight uninformative background textures that can actually impedevision.

SUMMARY

An image enhancement system may include a computer data processingsystem that includes at least one computer data hardware processor andother computer hardware components. The components of the system maycollectively perform various functions in connection with a modulationstrength map (MSM) that contains positive or negative valuesrepresenting a sign and magnitude of modulation to be applied in thevicinity of each of multiple locations in an image at each of multipleorientations for each of the locations, each of the multipleorientations at each of the multiple locations having an image value.The system may generate a net signed fractional modulation value at eachof the locations in the image using the MSM in combination with amodulation kernel F(θ). The generating may be done in accordance withthe following algorithm:

-   -   at a given location X in the image, and for a given orientation        θ, the modulation strength M represented in the MSM is used to        scale F(θ); and at every image location, positive and negative        contributions are accumulated from all scaled modulation kernels        arising from surrounding image locations/orientations, in order        to produce a net signed fractional modulation value that        represents a fractional increase or decrease to be applied to        the image value at that location.        The system may scale an image value at each location using the        net signed fractional modulation value at each location.        Application of the algorithm may not result in systematic        clipping of peri-edge locations to maximum or minimum brightness        values, or any other form of systematic homogenization of values        at peri-edge locations.

The modulation kernel may be an oriented bipolar function whose valueschange sign at the border of orientation θ that runs through the kernelcenter, but are primarily of the same sign on each side of the border.

The modulation kernel may be an oriented bipolar function whose valuesare peaked on either side of the border of orientation θ that runsthrough the kernel center, and decay smoothly moving perpendicularlyaway from the border.

The values of the modulation kernel may decay smoothly along anuninflected asymptotic curve moving perpendicularly away from theborder.

The modified image values produced by scaling the original image valuesby the net signed fractional modulation value may be clipped orcompressed so that no resulting value exceeds a maximum allowable imagevalue.

The system may generate the MSM from an edge strength map (ESM) thatrepresents edge strength at each of the multiple orientations at each ofthe multiple locations by non-monotonic mapping from edge strength tomodulation strength.

The signed fractional modulation values may be computed separately atmultiple spatial scales for the same image, after which the originalimage values are modified based on the modulation values combined acrossspatial scales.

The image may have different types of edges. A different set of ESMs,MSMs, and modulation kernels may be used in the scaling step to scalethe locations that comprise each different type of edge.

The value at each location may represent brightness, hue, or saturation.The scaling may scale the brightness, hue, or saturation values.

A non-transitory, tangible, computer-readable storage medium may containa program of instructions that may cause the computer data processingsystem to implement any combination of the functions described herein.

These, as well as other components, steps, features, objects, benefits,and advantages, will now become clear from a review of the followingdetailed description of illustrative embodiments, the accompanyingdrawings, and the claims.

BRIEF DESCRIPTION OF DRAWINGS

The drawings are of illustrative embodiments. They do not illustrate allembodiments. Other embodiments may be used in addition or instead.Details that may be apparent or unnecessary may be omitted to save spaceor for more effective illustration. Some embodiments may be practicedwith additional components or steps and/or without all of the componentsor steps that are illustrated. When the same numeral appears indifferent drawings, it refers to the same or like components or steps.

FIG. 1 is a flowchart illustrating an example of an image enhancementprocess.

FIGS. 2A-2D illustrate examples and perceptual effects of oriented,bipolar decaying modulation kernels for enhancing borders.

FIG. 3 illustrates examples of original images and for each, anassociated edge strength map (used directly as a modulation strength mapin the example), a modulation image, and an enhanced image.

FIG. 4 is an example of an image enhancement system.

FIG. 5 is an example of a storage medium containing a program ofinstructions.

DETAILED DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS

Illustrative embodiments are now described. Other embodiments may beused in addition or instead. Details that may be apparent or unnecessarymay be omitted to save space or for a more effective presentation. Someembodiments may be practiced with additional components or steps and/orwithout all of the components or steps that are described.

An image enhancement system may automatically modulate images so thatborder contrast is manipulated in systematic ways. One application maybe to enhance the salience of main object boundaries in images, whileleaving texture edges unaffected or suppressed. However, a great deal offlexibility exists to manipulate boundaries selectively both inmagnitude and direction, based on their orientation, strength, scale,and feature type.

Format of Input Images

FIG. 1 is a flowchart illustrating an example of an image enhancementprocess. An image enhancement system may implement this or a differentprocess. Examples of steps that may be followed in this process are nowdiscussed. A different set of steps or a different order may be usedinstead.

Image data may be loaded into an image enhancement system. The imagedata may be representative of an image 101 and may have originated fromany of several sources, such as a camera, document scanner, medicalimaging device, satellite imager, and/or another type of spatial sensorarray. The image data may be digitized versions of hand drawn pictures,or artificial images such as produced by computer graphics programs orcomputer-based graphical design tools. The image data may be arrays ofdata that have a spatial interpretation and include meaningful regionsseparated by boundaries.

The image may be an individual, still image or part of a series ofimages that form a video. Other images in the video may be processed inthe same way.

The image data may comprise values at multiple locations in the image,such as at each pixel in the image. The image data may contain one ormore channels of information, such as RGB (color) images ormultispectral images. The algorithms that are discussed below refer tomodulation of a single channel extracted from the image, such as theintensity channel, or some other channel which could be a raw channel(e.g. the red value in an RGB image) or a combination or transformationof original image channels, such as a red-green opponent channel or hueor saturation channels.

The examples discussed below refer to standard 3-channel color images.The intensity channel may be extracted and modulated, and the image maythen be reconstituted using the modified intensity channel. Had theimage instead been a greyscale image, the process may have operateddirectly on the image pixels, without need for the multi-channeldecomposition and recomposition steps.

Edge Strength Map (ESM)

As shown in FIG. 1, the image 101 in the form of image data may bereceived. Additionally, an edge strength map (ESM) 103 may be received.The ESM 103 may represent the edge strength at each of multipleorientations at each location in the image 101. The ESM 103 may have thesame width and height dimension as the original image, i.e. may have thesame number of image locations. The ESM 103 may include a depthdimension that accommodates multiple image-sized planes indexed by edgeorientation (θ).

FIG. 3 illustrates examples of original images and, for each, anassociated edge strength map (used directly as a modulation strength mapin the example), a modulation image, and an enhanced image. The edgestrength map is rendered by showing a pixel at each location whosedarkness level indicates the maximum value of edge strength in all theorientation planes associated with that pixel. In the examples ofprocessed images shown in FIG. 3, 16 orientation planes were used, from0 to 337.5° in equal steps of 22.5°. More or fewer orientation planescould have been used. More planes, if available, may lead to betterresults, as this may allow the edge detector to be better tuned inorientation. A similar point is made about having more image pixels,which allows the processing to occur at higher resolution. If too feworientation plan

The value stored in each entry in the ESM in the examples shown herewere calculated by a particular edge detection algorithm, andrepresented the probability ranging from 0 to 1 that an local objectboundary existed at that location and orientation in the image.Alternatively, the ESM value could have represented the strength of theresponse of any of a large number of different oriented edge detectionfilters, leading to similar results. The more reliable the edgedetection, and the more precise the tuning of the edge detector inposition and orientation, the better the image modulation results maybe.

Generating Modulation Strength Map (MSM)

The ESM 103 may represent the strength of the edge measurement at agiven location/orientation, or the strength of the belief than an edgeexists at that location/orientation, as discussed above.

A modulation strength map (MSM) 105 is a second data structure that maybe generated from the ESM 103. The MSM 105 may specify how much imagemodulation is to be applied to the image at each orientation at eachlocation. The MSM 105 can contain positive values, indicating that theedges at those particular locations/orientations should be boosted, thatis, the contrast across the oriented boundary should be increased. Itcan also contain negative values indicating that oriented contrast atthose locations/orientations should be suppressed.

In the example shown in FIG. 3, the MSM was equal to the ESM, so thatstronger edges were boosted proportionally more than weak edges and noedges were suppressed. Where the MSM is equal to the ESM, then theoriented edge information provided as input to the system can be viewedas having been directly loaded into the MSM. In this case, the ESM stageexists but is implicit: the existence of an MSM containing modulationvalues in a systematic relationship to edges present in the originalimage implies that an ESM exists, or an MSM containing values in asystematic relation to image edge locations/orientations could not havebeen generated.

In general, any number of mappings from an edge strength value E to amodulation strength value M could be used, including:

-   -   A linear function M=AE+B, where A is a scaling factor and B is        an offset    -   A more complex polynomial function such as M=AE²+BE+C, where A,        B, and C are constants, which would tend to exaggerate stronger        edges    -   A decelerating function such as M=√{square root over (E)}, such        that the modulation value begins to saturate for stronger edges    -   An S-shaped, or sigmoidal, function such as        M=1/(1+exp(−(E−t)/s), where t is a threshold and s is a        steepness parameter, which would leave weak edges relatively        unaffected, and saturate the modulation level for stronger        edges, or    -   A non-monotonic function of E, such as an upside-down U-shaped        function like sin(πE)−0.5, which would boost medium-strength        edges, while actively suppressing weak and strong edges.

In addition to these general mappings from E to M values, the mappingcan also take into account orientation. For example, positive modulationvalues could be assigned to vertical or near vertical edges, andnegative values could be assigned to horizontal or near horizontal edgesto emphasize vertical and de-emphasize horizontal structures in theimage.

Modulation Kernel

A modulation kernel 107 instantiated for each orientation maybeconvolved in a step 109 with the corresponding orientation plane of theMSM. The kernel represents the spatial pattern of boosting/suppressionthat may be effected centered at and in the vicinity of each location ateach orientation, scaled by the corresponding value of M. To achievethis, the modulation kernel 107 instantiated for each orientation θ isconvolved with the corresponding orientation plane of the MSM.

FIGS. 2A-2D illustrate examples and perceptual effects of oriented,bipolar decaying modulation kernels for enhancing borders. FIG. 2Eprovides examples of what has just been discussed. An equation is alsopresented below.

The results may then be summed across all orientations at each pixel ina step 107 to yield net signed fractional modulation values 111, one ateach pixel. Net positive values may indicate pixels whose values will beboosted, and net negative values may indicate pixels whose values willbe suppressed. These net signed fractional modulation values may then beadded to an image 115 containing all 1's in a step 113 to produce anunsigned modulation image 117 (where the value 1 indicates nomodulation) appropriate for scaling the original image values. Examplesof this are illustrated in FIG. 3C.

Modulation Kernels of Different Forms

In some applications, the purpose of the oriented modulation kernels maybe to alter the actual or perceived contrast across boundaries that havebeen detected in the image during the edge-detection pre-processingstage. Modulation kernels may in these applications be bipolar instructure, with positive coefficients that increase image values on oneside of the oriented border, and negative coefficients that reduce imagevalues on the other side of the border. If the kernel is signed suchthat positive coefficients are applied to the side of the border whoseimage values are initially larger, such as the brighter side of an edge,the effect of the kernel may be to brighten the bright side and darkenthe dark side of the edge. This may lead to a more prominent border andgreater perceived contrast between the surfaces abutting the two sidesof the edge. If the kernel is oppositely signed, such as effectivelyoccurs when the value of M is negative, then border contrast may bereduced by brightening the dark side of the edge and dimming the brightside. In this way, some borders can be enhanced, and others attenuated.

Based on the Cornsweet illusion (Cornsweet T. (1970) Visual Perception,New York: Academic Press, 1970), the kernel value may be bipolar, havingpeak positive and negative values immediately on either side of theedge, and decaying asymptotically on an exponential-like curve movingperpendicularly away from the edge. A 1-dimensional profile of this formis shown in FIG. 2A. The perceptual effect of modulating a border inthis way is illustrated in FIGS. 2B and 2C, in an illusion created by R.Beau Lotto. FIG. 2D shows 2-dimensional instantiations of the oriented,bipolar, decaying modulation kernel function used here for θ=0 and 45degrees, with varying settings. The function was defined as follows:

${F\left( {x,y} \right)} = \left\{ \begin{matrix}{A\; ^{{- {x}}S}^{{- y}\; D}} & {{W \geq x \geq 0};{{{- E}/2} \leq y \leq {E/2}}} \\{{- A}\; ^{{- {x}}S}^{- {yD}}} & {{{- W} \leq x < 0};{{{- E}/2} \leq y \leq {E/2}}}\end{matrix} \right.$

where,

A: Overall amplitude of the kernel

W: Width of the kernel in pixels in the direction perpendicular to theedge

S: Spatial decay constant perpendicular to the edge

E: Length of the kernel in pixels in the direction parallel to the edge

B: Spatial decay constant parallel to the length of the edge

The kernel was rotated in the plane to generate instantiations atdifferent values of q.

Modulating Image

In the examples shown in FIG. 3, the modulation image multiplied theintensity channel of the original image to produce the enhanced imageshown. Where the multiplied values exceeded the maximum allowable imagevalue, the values were clipped to the maximum. This multiplication isshown in FIG. 1 as a step 119, yielding an enhancement image 121.

A variant of this approach may eliminate the need for clipping forvalues that would otherwise exceed the maximum allowable value. Withthis approach, the modified image value resulting from the scalingoperation may be passed through a compressive nonlinearity so that themaximum allowable image value is never exceeded.

In general, the operation used to modulate image values using the valuein the modulation image should share the property with multiplicationthat larger image values are generally modulated more than smaller imagevalues up to the limits imposed by having a maximum allowable imagevalue, but where a direct proportionality between the original imagevalue and the modulated image value is not required.

Variations

The image processing approach described above may be applied to a singletype of edge feature at a single spatial scale. The approach mayincorporate:

-   -   Multiple edge types. A different ESM can be provided for each        different type of edge in an image. Each different ESM may        utilize a scheme that is optimized for its type of edge. Each        different ESM may be processed with a correspondingly different        MSM and modulation kernel. This allows for different modulation        strategies for different types of edges, such as occluding        boundaries, planar boundaries, and shadow boundaries. For        example, it may be desirable to enhance occluding boundaries and        planar boundaries, but to suppress shadow boundaries. Similarly,        different ESMs can be provided for edges with different local        curvature, so that straight edges can be modulated differently        than curved edges, both by using different values in their        respective MSMs, and by using differently shaped modulation        kernels that track the edge curvature. For example, in certain        types of images, this may help emphasize architectural        structures while suppressing natural foliage.    -   More complex feature types. Features such as L-junctions that        conjoin two local edges at different orientations, or more        complex features that conjoin more edge components, can be        handled by including more parameter planes that are indexed by        multiple θ, parameters rather than just one. Modulation kernels        may also be specialized in shape for different complex feature        types, as they would be for curves (see multiple edge types        explanation above), so that border contrast is appropriately        modulated.    -   Modulation at multiple scales. Natural boundaries may have a        different structure at different spatial scales. For example,        the outline of a tree typically has one orientation at a coarse        scale (of the canopy), and different, rapidly varying        orientations at a fine scale (of leaves). The above described        approach to modulating pixels based on edge information allows        for modulations to be combined across scales, such as by        modulating the original image separately and serially, one scale        at a time. It would thus be possible, for example to increase        the salience of the canopy contour, while attenuating the        detailed leaf contours.

FIG. 4 is an example of an image enhancement system 401. As illustratedin FIG. 4, the image enhancement system 401 may include a dataprocessing system 403 containing one or more computer data hardwareprocessors 405 and other computer hardware components 407, such as oneor more tangible memories (e.g., random access memories (RAMs),read-only memories (ROMs), and/or programmable read only memories(PROMS)), tangible storage devices (e.g., hard disk drives, CD/DVDdrives, and/or flash memories), system buses, video processingcomponents, network communication components, input/output ports, and/oruser interface devices (e.g., keyboards, pointing devices, displays,microphones, sound reproduction systems, and/or touch screens).

The data processing system 403 may be configured to perform one, all, orany combination of the functions that have been described above and/orin the claims below, such as the process illustrated in FIG. 1 anddescribed above.

The data processing system 403 may be a desktop computer or a portablecomputer, such as a laptop computer, a notebook computer, a tabletcomputer, a PDA, a smartphone, or part of a larger system, such avehicle, appliance, and/or telephone. The data processing system mayinclude one or more computers at the same or different locations. Whenat different locations, the computers may be configured to communicatewith one another through a wired and/or wireless network communicationsystem.

The data processing system 403 may include software (e.g., one or moreoperating systems, device drivers, application programs, and/orcommunication programs). When software is included, the softwareincludes programming instructions and may include associated data andlibraries. When included, the programming instructions are configured toimplement one or more algorithms that implement one or more of thefunctions of the data processing system 403, as recited herein. Thedescription of each function that is performed by the data processingsystem 403 also constitutes a description of the algorithm(s) thatperforms that function.

The software may be stored on or in one or more non-transitory, tangiblestorage devices, such as one or more hard disk drives, CDs, DVDs, and/orflash memories. The software may be in source code and/or object codeformat. Associated data may be stored in any type of volatile and/ornon-volatile memory. The software may be loaded into a non-transitorymemory and executed by one or more processors.

FIG. 5 is an example of a non-transitory, tangible, computer-readablestorage medium 501 containing a program of instructions 503. The programof instructions may cause the data processing system 503 to perform thefunctions of the data processing system 503 as described above, whenloaded in and run by the data processing system 503.

The components, steps, features, objects, benefits, and advantages thathave been discussed are merely illustrative. None of them, nor thediscussions relating to them, are intended to limit the scope ofprotection in any way. Numerous other embodiments are also contemplated.These include embodiments that have fewer, additional, and/or differentcomponents, steps, features, objects, benefits, and advantages. Thesealso include embodiments in which the components and/or steps arearranged and/or ordered differently.

For example, the system may not use very high magnitude modulationvalues and or values in the modulation kernel, such that, where edgesare indicated to exist in the ESM, original image values aresystematically scaled past their allowable limits at the modulationstep, so that, after clipping to maximum allowed values (for example,255 in an 8-bit digital image), or rounding to the nearest value on thelow end (for example, 0 in an 8-bit digital image) the resulting valuessurrounding the edge are homogenized to the minimum and maximumallowable values. In this configuration, the original image values mayhave effectively been overwritten. If 0 is interpreted as black, and 255is interpreted as white, then the perceptual effect may be to haveapplied a black line or band on one side of the edge and a white line orband on the other side. Similarly, if a way were found to use themodulation kernel and modulation values so that image values in thevicinity of edges were systematically homogenized to other values thatwere not the minimum or maximum values, so that original image pixels inthe vicinity of edges were effectively replaced by “paint” of differentvalues, such as 100 and 200 in the above example, this might also beexcluded.

Unless otherwise stated, all measurements, values, ratings, positions,magnitudes, sizes, and other specifications that are set forth in thisspecification, including in the claims that follow, are approximate, notexact. They are intended to have a reasonable range that is consistentwith the functions to which they relate and with what is customary inthe art to which they pertain.

All articles, patents, patent applications, and other publications thathave been cited in this disclosure are incorporated herein by reference.

The phrase “means for” when used in a claim is intended to and should beinterpreted to embrace the corresponding structures and materials thathave been described and their equivalents. Similarly, the phrase “stepfor” when used in a claim is intended to and should be interpreted toembrace the corresponding acts that have been described and theirequivalents. The absence of these phrases from a claim means that theclaim is not intended to and should not be interpreted to be limited tothese corresponding structures, materials, or acts, or to theirequivalents.

The scope of protection is limited solely by the claims that now follow.That scope is intended and should be interpreted to be as broad as isconsistent with the ordinary meaning of the language that is used in theclaims when interpreted in light of this specification and theprosecution history that follows, except where specific meanings havebeen set forth, and to encompass all structural and functionalequivalents.

Relational terms such as “first” and “second” and the like may be usedsolely to distinguish one entity or action from another, withoutnecessarily requiring or implying any actual relationship or orderbetween them. The terms “comprises,” “comprising,” and any othervariation thereof when used in connection with a list of elements in thespecification or claims are intended to indicate that the list is notexclusive and that other elements may be included. Similarly, an elementpreceded by an “a” or an “an” does not, without further constraints,preclude the existence of additional elements of the identical type.

None of the claims are intended to embrace subject matter that fails tosatisfy the requirement of Sections 101, 102, or 103 of the Patent Act,nor should they be interpreted in such a way. Any unintended coverage ofsuch subject matter is hereby disclaimed. Except as just stated in thisparagraph, nothing that has been stated or illustrated is intended orshould be interpreted to cause a dedication of any component, step,feature, object, benefit, advantage, or equivalent to the public,regardless of whether it is or is not recited in the claims.

The abstract is provided to help the reader quickly ascertain the natureof the technical disclosure. It is submitted with the understanding thatit will not be used to interpret or limit the scope or meaning of theclaims. In addition, various features in the foregoing detaileddescription are grouped together in various embodiments to streamlinethe disclosure. This method of disclosure should not be interpreted asrequiring claimed embodiments to require more features than areexpressly recited in each claim. Rather, as the following claimsreflect, inventive subject matter lies in less than all features of asingle disclosed embodiment. Thus, the following claims are herebyincorporated into the detailed description, with each claim standing onits own as separately claimed subject matter.

The invention claimed is:
 1. An image enhancement system comprising acomputer data processing system that includes at least one computer datahardware processor and other computer hardware components that, inconnection with a modulation strength map (MSM) that contains positiveor negative values representing a sign and magnitude of modulation to beapplied in the vicinity of each of multiple locations in an image ateach of multiple orientations for each of the locations, each of themultiple orientations at each of the multiple locations having an imagevalue, collectively: generate a net signed fractional modulation valueat each of the locations in the image using the MSM in combination witha modulation kernel F(θ), the generating being done in accordance withthe following algorithm: at a given location X in the image, and for agiven orientation θ, the modulation strength M represented in the MSM isused to scale F(θ); at every image location, positive and negativecontributions are accumulated from all scaled modulation kernels arisingfrom surrounding image locations/orientations, in order to produce a netsigned fractional modulation value that represents a fractional increaseor decrease to be applied to the image value at that location; and scalean image value at each location using the net signed fractionalmodulation value at each location, wherein application of the algorithmdoes not result in systematic clipping of peri-edge locations to maximumor minimum brightness values, or any other form of systematichomogenization of values at peri-edge locations.
 2. The imageenhancement system of claim 1 wherein the modulation kernel is anoriented bipolar function whose values change sign at the border oforientation θ that runs through the kernel center, but are primarily ofthe same sign on each side of the border.
 3. The image enhancementsystem of claim 2 wherein the modulation kernel is an oriented bipolarfunction whose values are peaked on either side of the border oforientation θ that runs through the kernel center, and decay smoothlymoving perpendicularly away from the border.
 4. The image enhancementsystem of claim 3 wherein the values of the modulation kernel decaysmoothly along an uninflected asymptotic curve moving perpendicularlyaway from the border.
 5. The image enhancement system of claim 1 whereinthe modified image values produced by scaling the original image valuesby the net signed fractional modulation value are clipped or compressedso that no resulting value exceeds a maximum allowable image value. 6.The image enhancement system of claim 1 wherein the system generates theMSM from an edge strength map (ESM) that represents edge strength ateach of the multiple orientations at each of the multiple locations bynon-monotonic mapping from edge strength to modulation strength.
 7. Theimage enhancement system of claim 1 wherein the signed fractionalmodulation values are computed separately at multiple spatial scales forthe same image, after which the original image values are modified basedon the modulation values combined across spatial scales.
 8. The imageenhancement system of claim 1 wherein: the system generates the MSM froman edge strength map (ESM) that represents edge strength at each of themultiple orientations at each of the multiple locations; the image hasdifferent types of edges; and a different set of ESMs, MSMs, andmodulation kernels are used in the scaling step to scale the locationsthat comprise each different type of edge.
 9. The image enhancementsystem of claim 1 wherein: the value at each location representsbrightness, hue, or saturation; and the scaling scales the brightness,hue, or saturation values.
 10. A non-transitory, tangible,computer-readable storage medium containing a program of instructionsthat cause a computer data processing system that includes at least onecomputer data hardware processor and other computer hardware componentsrunning the program of instructions, in connection with a modulationstrength map (MSM) that contains positive or negative valuesrepresenting a sign and magnitude of modulation to be applied in thevicinity of each of multiple locations in an image at each of multipleorientations for each of the locations, each of the multipleorientations at each of the multiple locations having an image value,to: generate a net signed fractional modulation value at each of thelocations in the image using the MSM in combination with a modulationkernel F(θ), the generating being done in accordance with the followingalgorithm: at a given location X in the image, and for a givenorientation θ, the modulation strength M represented in the MSM is usedto scale F(θ); in the scaled result, centered at location X, positivevalues indicate locations relative to X that are to be increased invalue and by how much, and negative values indicate image locationsrelative to X that are to be decreased in value and by how much; and atevery image location, these positive and negative contributions areaccumulated from all of the scaled modulation kernels arising fromsurrounding image locations/orientations, in order to produce a netsigned fractional modulation value that represents the fractionalincrease or decrease to be applied to the image value at that location;and scale an image value at each location using the net signedfractional modulation value at each location, wherein application of thealgorithm does not result in systematic clipping of peri-edge locationsto maximum or minimum brightness values, or any other form of systematichomogenization of values at peri-edge locations.
 11. The storage mediumof claim 10 wherein the modulation kernel is an oriented bipolarfunction whose values change sign at the border of orientation θ thatruns through the kernel center, but are primarily of the same sign oneach side of the border.
 12. The storage medium of claim 11 wherein themodulation kernel is an oriented bipolar function whose values arepeaked on either side of the border of orientation θ that runs throughthe kernel center, and decay smoothly moving perpendicularly away fromthe border.
 13. The storage medium of claim 12 wherein the values of themodulation kernel decay smoothly along an uninflected asymptotic curvemoving perpendicularly away from the border.
 14. The storage medium ofclaim 10 wherein the modified image values produced by scaling theoriginal image values by the net signed fractional modulation value areclipped or compressed so that no resulting value exceeds a maximumallowable mage value.
 15. The storage medium of claim 10 wherein thesystem generates the MSM from an edge strength map (ESM) that representsedge strength at each of the multiple orientations at each of themultiple locations by non-monotonic mapping from edge strength tomodulation strength.
 16. The storage medium of claim 10 wherein thesigned fractional modulation values are computed separately at multiplespatial scales for the same image, after which the original image valuesare modified based on the modulation values combined across spatialscales.
 17. The storage medium of claim 10 wherein: the system generatesthe MSM from an edge strength map (ESM) that represents edge strength ateach of the multiple orientations at each of the multiple locations; theimage has different types of edges; and a different set of ESMs, MSMs,and modulation kernels are used in the scaling step to scale thelocations that comprise each different type of edge.
 18. The storagemedium of claim 10 wherein: the value at each location representsbrightness, hue, or saturation; and the scaling scales the brightness,hue, or saturation values.